This invention relates to the field of 3D displays.
Autostereoscopic 3D displays are a well-known class of 3D display that can be viewed without needing to wear special glasses. The general principle is to ensure that the image seen on a display screen is dependant on the position of the viewer. At least two images are usually used, showing the desired view of the scene from two positions, corresponding to the positions of the left and right eye of the viewer for a limited range of head positions. The images are usually created by photographing a scene from positions corresponding to the centres of the intended eye positions. More than two images can be used, which allows a greater degree of head movement, whilst showing changes of parallax in the image, leading to increased realism. The images used for such a so-called multiview display are usually acquired by several cameras, each positioned so as to capture a view of the scene appropriate for the eye position at which the image is visible.
As an alternative to using several cameras, images may be generated using computer graphics techniques to render images equivalent to those that would be captured by a two or more real cameras. The process of rendering an image to simulate an image captured by a real camera is known as perspective projection. Images rendered in this way must of course exhibit the usual properties of images obtained from a conventional camera, specifically that objects appear smaller when they are a long way away, and the angle at which the scene is viewed varies slightly across the image, by an amount equal to the angular width and height of the image.
There are a number of known methods for realising an autostereoscopic display. For example, a sheet of lenticular lenses can be placed over a high-resolution display. The image underneath each lenticular lens is formed by taking one column of pixels from each image in turn, so that the display presents the viewer with each image in sequence as the viewer""s head moves parallel to the image. This process is illustrated in FIG. 2. An example of this xe2x80x9cmultiviewxe2x80x9d approach is described in xe2x80x9cDesign and applications of multiview 3D-LCDxe2x80x9d by C. van Berkel et al, 1996 EuroDisplay Conference, Birmingham, October 1996, and xe2x80x9cCharacterisation and optimisation of 3D-LCD module designxe2x80x9d by C. van Berkel and J. Clarke, SPIE International Conference on Electronic Imaging, San Jose, Feb. 11-14, 1997. A set of conventional cameras is used to capture the images.
However, autostereoscopic displays based on these principles do not produce a fully accurate 3D representation of the scene; they merely presents the viewer with a number of discrete views.
One problem is that often, the viewer will be sufficiently close to the screen that one of his eyes is presented with parts of one image from microlenses on the left of the screen and parts of another image from microlenses on the right, due to the changing angle between the screen and the direction of view. These combined views will not represent an accurate portrayal of the scene. This is illustrated in FIG. 4. It is possible to minimise this effect by careful design of the display system to account for the variation of angle for a given viewing distance. An example of such a design is described in UK patent Application 2,272,597A (Sharp), which describes an arrangement of illumination elements and cylindrical lenses which ensure that a given observer sees a single view across the whole of the display. However, this will only work for observers at a certain range of distances.
There is a known method of generating an autostereoscopic display which makes, no assumptions about the position of the viewer, and is capable of reproducing a 3D scene with high accuracy. This method is known as Integral Imaging, an example of which is described by McCormick et al., xe2x80x9cExamination of the requirements for autostereoscopic, full parallax, 3D TVxe2x80x9d, IBC ""94, IEE Conference Publication 397, September 1994, pp.477-482, and WO-A-9534018. In this method, a special kind of camera captures an image which is xe2x80x98replayedxe2x80x99 by placing a sheet of microlenses over the image. The camera consists of an arrangement of lenses, which can be thought of as behaving like an array of very small cameras, one positioned at the location of each microlens. When the image is replayed, each microlens appears to have a colour and brightness that is a function of the angle at which it is viewed. The form of the image underneath the microlens array is such that the overall appearance is of a 3D image. The microlens array should ideally be composed of spherical microlenses, although lenticular lenses can also be used, in which case the image only exhibits parallax in the direction normal to the lenticular elements.
It is known that the image underneath the microlens array can be generated by a computer, rather than by a special kind of camera. This allows a 3D image of a computer model to be generated. However, techniques for doing this have involved ray tracing and simulation of the behaviour of the integral image camera, and are therefore complex and processor-intensive. For example, the image underneath each microlens can be computed by rendering an image of the scene from the viewpoint of that microlens, with an appropriate modification to allow objects that are on both sides of the image plane to be visible. For a typical microlens array consisting of, say 400 by 300 microlenses, this would require 120,000 separate rendering processes.
The present invention seeks to providexe2x80x94in one of its formsxe2x80x94a method of allowing conventional image rendering hardware to be used efficiently to produce an image to place underneath the microlens array, whilst enabling an accurate 3D representation of the scene to be provided.
Accordingly, the present invention consists, in one aspect, in a method of providing a viewable representation of a 3D model, comprising the steps of identifying a set of different viewing angles and rendering from said model a set of orthographic projections corresponding respectively with said viewing angles.
It is important to note that the present invention uses orthographic projections. An image rendered using orthographic projection is unlike an image captured with a conventional camera; indeed it is impossible to capture such an image with a conventional camera. One fundamental difference is that the size of an object in an orthographic image projection is independent of its distance from the camera; in conventional perspective projection (or in a normal camera image), more distant objects appear smaller (the so-called perspective effect). This difference comes about because the angle at which the scene is viewed in an orthographic image is constant across the whole image. The field-of-view is specified in terms of the width and height of the rendered image (measured in units of distance, e.g. feet/meters), rather than by a horizontal and vertical field of view (measured in angular units e.g. degrees).
We have found that using orthographic images overcomes a major problem found with conventional multiview displays. Specifically, in such displays, as explained earlier, the property that determines which image is visible from a particular viewing position at any point on the screen is the angle between the screen and the direction of view of that point, as shown in FIG. 4. This will tend to result in several views being seen across the width of the display, which is an undesirable property if each view corresponds to a normal perspective camera image. As already mentioned, some displays have proposed modifications to the basic arrangements in an attempt to take account of the changing viewing angle across the display, for example the use of varying lens offsets across the display, as described for example in GB 2,272,597A.
In direct contrast to such complex prior methods of compensating for problems with conventional perspective images, the present invention uses orthographic images. This might at first seem unsatisfactory as the viewer should be presented with a xe2x80x9cnaturalxe2x80x9d perspective image. However, it has been appreciated according to the invention that the nature of a display will take account of the changing viewing angle of the viewer with respect to the display. Specifically, the display will allow portions of different orthographically-rendered images to be visible across the display from a given viewing point. Since the viewing angle is constant across each orthographic image, but differs from one orthographic image to another, the changing viewing angle automatically provides the correct parameter to control the selection of which image is seen. It has been appreciated that this is precisely what is needed to create a natural (perspective) image from the viewpoint of an observer, tailored to the particular location of the observer. Thus, although the individual images rendered are orthographic, what the viewer will actually see will appear to have the correct perspective.
A further advantage is that the orthographic projections may be rendered efficiently. Firstly, orthographic projections may be simpler to perform than perspective projections, as the distance from the source does not need to be taken into account. In a preferred arrangement, a greater benefit is that the number of rendering processes may be equal to the number of viewpoints, typically equal to the number of pixels under each micro lens, and this will normally be smaller than the total number of lenses. For example, in a lenticular display with a resolution of 1024xc3x97768 and 64 horizontal viewing angles, nearly a million rendering operations would be required with conventional processing but only 64 rendering operations may be required with an embodiment, a saving of better than 10,000 fold. It will be appreciated, as described below, that each rendering operation may comprise multiple rendering processes, to take into account depth of field.
A yet further advantage that may be attained in preferred embodiments is that orthographic projection may allow depiction of objects both behind and in front of the display plane whereas conventional perspective rendering normally can only display objects on one side of the display. This advantageous feature may be provided independently.
Advantageously, the set of orthographic projections are spatially multiplexed to provide a set of micro images, each micro image containing one pixel from each of the projections.
In another aspect, the present invention consists in representation of a 3D scene or model for display utilising 2D display means, comprising a set of 2D images representing respective orthographic projections of the 3D scene or model at respective different angles.
In still another aspect, the present invention consists in a method of processing a viewable representation of a 3D model or scene, comprising the steps of demultiplexing the representation to form a set of orthographic projections corresponding respectively with different viewing angles, and processing each projection.
The invention extends to software and hardware for implementing the aspects and preferred features. Further aspects and preferred features are set out in the claims and/or in the following specific description.
Advantageously, the processed projections are remultiplexed.
The present invention recognises that each pixel underneath a spherical microlens in a true integral image represents the brightness and colour of the scene when viewed from the position of that microlens, at an angle that is a function of the position of the pixel with respect to the focal point of the microlens. We will assume for now that the arrangement of pixels underneath each microlens are the same for all microlenses. Therefore, for example, the top left-hand pixel underneath each microlens corresponds to the light ray leaving the scene at a given angle, and this angle is the same for all top left-hand pixels under all microlenses. The position at which the light ray intersects the display surface varies according to the position of the microlens, but the angle does not vary. Thus, if we were to take the top left-hand pixel from under each microlens and assemble them into an image, we would obtain a kind of image of the scene, similar to that which would be obtained by a camera a very long way away, having a very narrow field-of-view, such that the angle of view was essentially unchanging across the image.
An embodiment of the invention will now be described by way of example with reference to the accompanying drawings.